Web Survey Bibliography
Measurement effects are a major problem in mixed-mode surveys suggesting that the same respondent potentially provides different answers under different modes. Mixed-mode researchers therefore often need to know the average size of measurement effects (AME) for the questions of their interest. The present paper discusses estimation of AME using two different data collection approaches: a between subject and a within subject (repeated measures) design. Real-world data from an experiment with N=8,800 subjects in The Netherlands are presented. In the ‘between design’, subjects were randomly allocated to one mode only (Face-to-Face, Telephone, Mail, or Web). In the ‘within design’ subjects were first allocated as in the ‘between design’ and subsequently re-approached after some weeks in a reference mode (Face-to-Face) repeating a large number of questions. Unit nonresponse in both designs represents a threat to full randomization and thus to unbiased estimation of the AME, if confounders relate to the selection mechanism into mode conditions and the outcome variable. Statistical adjustment of missing data is a possible solution to this problem, but it is based on assumptions. Adjustment in ‘between designs’ assumes that the selection mechanism is ignorable given auxiliary variables. This is often contestable in practice, because some important confounders might not be observed. An advantage of ‘within designs’ is that it is more plausible to ignore the selection mechanism when conditioning on the repeated measurements. Thereby it is not problematic whether time-related changes of outcomes between measurement occasions occur, because these can be controlled using subjects who are allocated to the reference mode on both occasions (i.e., Face-to-Face). However, ‘within designs’ need to assume that measurements can be taken independently across time. We compare AME estimates from both designs for questions from the Dutch Crime Victimization Survey applying regression adjustment with propensity score strata as covariates or propensity score weighting.
Conference Homepage (abstract)
Web survey bibliography - Schouten, B. (20)
- Adaptive survey designs to minimize survey mode effects – a case study on the Dutch Labor Force...; 2016; Calinescu, M.; Schouten, B.
- Tailored fieldwork design to increase representative household survey response: an experiment in the...; 2015; Luiten, A.; Schouten, B.
- Coding Surveys on their Item Characteristics: Reliability Diagnostics; 2015; Bais, F.; Schouten, B.; Toepoel, V.
- Selection error in single- and mixed mode surveys of the Dutch general population; 2015; Hox, J., Klausch, L. T., Schouten, B.
- Evaluating mixed-mode redesign strategies against benchmark surveys: the case of the Crime Victimization...; 2014; Klausch, L. T., Hox, J., Schouten, B.
- The impact of contact effort on mode-specific selection and measurement bias; 2014; Schouten, B., van der Laan, J., Cobben, F.
- The use of within-subject experiments for estimating measurement effects in mixed-mode surveys ; 2014; Klausch, L. T., Schouten, B., Hox, J.
- The impact of contact effort and interviewer performance on mode-specific nonresponse and measurement...; 2014; Schouten, B., Cobben, F., van der Laan, J., Arends, J.
- Measurement Effects of Survey Mode on the Equivalence of Attitudinal Rating Scale Questions; 2013; Klausch, L. T., Hox, J., Hox, J., Schouten, B.
- Adaptive survey designs to minimize survey mode effects. A case study on the Dutch Labour Force Survey...; 2013; Calinescu, M., Schouten, B.
- Optimizing quality of response through adaptive survey designs; 2013; Schouten, B., Calinescu, M., Luiten, A.
- Mode Effects in Mixed-Mode Surveys: Prevention, Diagnostics, and Adjustment 1; 2013; de Leeuw, E. D., Dillman, D. A., Schouten, B.
- Estimating Measurement Effects of Survey Modes From Between and Within Subject Designs; 2013; Klausch, L. T., Hox, J., Schouten, B.
- Mixed-mode including web: Recent developments at Statistics Netherlands; 2013; Luiten, A., Schouten, B.
- Disentangling Mode-Specific Selection and Measurement Bias in Social Surveys; 2012; Buelens, B., van der Laan, J., Schouten, B., Klausch, L. T., van der Brakel, J., Burger, J.
- Assessing Measurement Equivalence and Bias of Questions in Mixed-mode Surveys Under Controlled Sample...; 2012; Klausch, L. T., Hox, J., Schouten, B.
- The Representativity of Web Surveys of the General Population compared to Traditional Modes and Mixed...; 2012; Klausch, L. T., Schouten, B., Hox, J.
- Disentangling relative mode effects for the web survey mode in the Safety Monitor; 2011; Schouten, B., van de Brakel, J., Buelens, B., Klausch, L. T., van der Laan, J.
- Indicators for the representativeness of survey response; 2009; Schouten, B., Cobben, F., Bethlehem, J.
- Nonresponse Bias in Surveys; 2009; Bethlehem, J., Vehovar, V., Stoop, I., Schouten, B., Shlomo, N., Skinner, C., Montaquila, J.